Passive three-dimensional face imaging based on macro-structure and micro-structure image sizing

ABSTRACT

Techniques are described for passive three-dimensional (3D) face imaging based on macro-structure and micro-structure image sizing, such as for biometric facial recognition. A set of images of a user&#39;s face is processed to extract authentication deterministic macro-structure (DMAS) measurements. A database includes profile DMAS measurements, profile location definitions for deterministic micro-structure (DMIS) feature regions, and profile DMIS signatures computed for the DMIS feature regions. A first-level authentication determination can be based on comparing the authentication DMAS measurements with the profile DMAS measurements. Authentication DMIS signatures can be computed from sub-images obtained for the DMIS feature regions at the profile location definitions. A second-level authentication determination can be based on comparing the authentication DMIS signatures with the profile DMIS signatures. An authentication result can be output based on both the first-level authentication determination and the second-level authentication that indicates whether authentication of the user is granted or denied.

FIELD

The invention relates generally to optics integrated into personalelectronic devices. More particularly, embodiments relate to passivethree-dimensional face imaging based on macro-structure andmicro-structure image sizing, such as for biometric facial recognition.

BACKGROUND

Many modern electronic devices, such as smart phones, tablets, andlaptops, are equipped with biometric security access systems, such asface identification (ID), fingerprint sensors, and the like. Forexample, face ID may be used to unlock a smart phone, log in toapplications and accounts, to authorize mobile payments, etc. Similarface ID techniques are integrated into other access-control devices,such as electronic locks and automated teller machines (ATMs). Effectiveimplementation designs tend to balance various considerations. Forexample, it is typically desirable to provide rapid and accurate resultsto a user in a manner that avoids both false positives (which reduce thesecurity of the implementation) and false negatives (which can befrustrating to authorized users).

Conventional face ID systems tend to include relatively rapid, butrelatively insecure facial recognition approaches based onidentification of a limited number of large-scale structures. Suchapproaches tend to be relatively easy to spoof, for example, by using atwo-dimensional image of an authorized user's face, a three-dimensionalwax or latex model of an authorized user's face, or the like. Forexample, conventional face ID implementations on smart phones aretypically designed to minimize usage of battery resources, memoryresources, processor resources, etc. Also, conventional face IDimplementations on smart phones tend not to be overly concerned withadvanced spoofing techniques, or the like; and they tend to err on theside of allowing more false positives than false negatives to avoidfrustrating authorized users who are trying to quickly unlock theirsmart phones. However, for many smart phone and other applications, itcan be desirable to provide a higher level of security (includingadditional protection against spoofing), without excessively impactingbattery, memory, processor, and other resources.

BRIEF SUMMARY OF THE INVENTION

Embodiments provide passive three-dimensional (3D) face imaging based onmacro-structure and micro-structure image sizing, such as for biometricfacial recognition. For example, an imaging system (e.g., in a smartphone or other electronic device) can be used to capture a set of imagesof a user's face. The set of images can be processed to extractauthentication deterministic macro-structure (DMAS) measurements, suchas measurements of eyes, nose, ear, and other large-scale features thatdo not appreciably change relative shape, size, location, orientation,etc. over time. A registration profile can be retrieved from a face IDdatabase, which includes profile DMAS measurements, profile locationdefinitions for deterministic micro-structure (DMIS) feature regions,and profile DMIS signatures computed for the DMIS feature regions. Afirst-level authentication determination can be made based on comparingthe authentication DMAS measurements with the profile DMAS measurements.Characteristic sub-images can be obtained for the DMIS feature regionsbased on the profile location definitions, and authentication DMISsignatures can be computed from the characteristic sub-images. Asecond-level authentication determination can be made based on comparingthe authentication DMIS signatures with the profile DMIS signatures. Anauthentication result can be output based on both the first-levelauthentication determination and the second-level authentication thatindicates whether authentication of the user is granted or denied.

According to one set of embodiments, a method is provided forauthentication of a user based on passive face imaging. The methodincludes: capturing, by an imaging system, a set of images of a user'sface; processing the set of images to extract authenticationdeterministic macro-structure (DMAS) measurements; generating afirst-level authentication determination based on comparing theauthentication DMAS measurements with profile DMAS measurements;obtaining one or more characteristic sub-images of one or moredeterministic micro-structure (DMIS) feature regions based on profilelocation definitions for each of the one or more DMIS feature regions;computing one or more authentication DMIS signatures from the one ormore characteristic sub-images; generating a second-level authenticationdetermination based on comparing the authentication DMIS signatures withprofile DMIS signatures computed for each of the one or more DMISfeature regions; and outputting an authentication result based on boththe first-level authentication determination and the second-levelauthentication that indicates whether authentication of the user isgranted or denied.

According to another set of embodiments, a system is provided forauthentication of a user based on passive face imaging. The systemincludes: an imaging camera to capture a set of images of a user's face;a face identification database having, stored thereon, a registrationprofile comprising profile deterministic macro-structure (DMAS)measurements, profile location definitions for each of one or moredeterministic micro-structure (DMIS) feature regions, and at least oneprofile DMIS signature computed for each of the one or more DMIS featureregions; and a control and processing module having one or moreprocessors, and a memory having, stored thereon, a set of instructions.Executing the set of instructions causes the one or more processors,operating in a user authentication mode, to: process the set of imagesto extract authentication DMAS measurements; generate a first-levelauthentication determination based on comparing the authentication DMASmeasurements with the profile DMAS measurements; obtain one or morecharacteristic sub-images of the one or more DMIS feature regions basedon the profile location definitions for each of the one or more DMISfeature regions; compute one or more authentication DMIS signatures fromthe one or more characteristic sub-images; generate a second-levelauthentication determination based on comparing the authentication DMISsignatures with the profile DMIS signatures; and output anauthentication result based on both the first-level authenticationdetermination and the second-level authentication that indicates whetherauthentication of the user is granted or denied. In some embodiments,executing the set of instructions causes the one or more processorsfurther, operating in a registration mode, to: direct capturing, by theimaging camera, of a set of profile images of an authorized user's face;process the set of profile images to extract the profile DMASmeasurements and to identify the one or more profile DMIS featureregions; for each profile DMIS feature region of the one or more profileDMIS feature regions: generate a respective profile location definitionbased on the profile DMAS measurements; obtain one or more respectivecharacteristic profile sub-images from image data of the profile DMISfeature region; and compute one or more respective profile DMISsignatures from the one or more respective characteristic profilesub-images; and store, as a registration profile for the authorized userin the face identification database, at least some of the profile DMASmeasurements, at least some of the respective profile locationdefinitions for the profile DMIS feature regions, and at least some ofthe profile DMIS signatures computed for the profile DMIS featureregions.

According to another set of embodiments, a method is provided forregistration of an authorized user to support subsequent faceauthentication based on passive face imaging. The method includes:capturing, by an imaging system, a set of images of an authorized user'sface; processing the set of images to extract deterministicmacro-structure (DMAS) measurements; storing at least some of the DMASmeasurements as part of a registration profile in a face identification(ID) database; processing the images further to identify one or moredeterministic micro-structure (DMIS) feature regions; and for each DMISfeature region of the one or more DMIS feature regions: generating arespective location definition based on the DMAS measurements; obtainingone or more respective characteristic sub-images from image data of theDMIS feature region; computing one or more respective DMIS signaturesfrom the one or more respective characteristic sub-images; and storing,further as part of the registration profile in the face identification(ID) database, the respective location definition for the DMIS featureregion, and at least one of the one or more respective DMIS signaturescomputed for the DMIS feature region.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, referred to herein and constituting a parthereof, illustrate embodiments of the disclosure. The drawings togetherwith the description serve to explain the principles of the invention.

FIG. 1 illustrates a mobile device with an integrated imaging system.

FIG. 2 illustrates a simplified block diagram of an electronic accesscontrol system using face ID sensing according to some embodiments.

FIG. 3 shows an imaging environment to illustrate the optical principlesinvolves in using image size to derive depth information from imaging,as a context for embodiments herein.

FIG. 4 shows an illustrative image of a face with various deterministicmacro-structures.

FIG. 5 shows an illustrative set of imaging data as a context forvarious embodiments described herein.

FIG. 6 shows illustrative partial chromatic responses for illustrativeportions of two characteristic sub-images.

FIG. 7 shows a flow diagram of an illustrative method for registrationof a user, according to various embodiments herein.

FIG. 8 shows a flow diagram of an illustrative method for authenticationof a user, according to various embodiments herein.

FIG. 9 shows a flow diagram of an example gated process for accesscontrol, according to various embodiments.

FIG. 10 provides a schematic illustration of one embodiment of acomputer system that can implement various system components and/orperform various steps of methods provided by various embodiments.

In the appended figures, similar components and/or features can have thesame reference label. Further, various components of the same type canbe distinguished by following the reference label by a second label thatdistinguishes among the similar components. If only the first referencelabel is used in the specification, the description is applicable to anyone of the similar components having the same first reference labelirrespective of the second reference label.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, numerous specific details are provided fora thorough understanding of the present invention. However, it should beappreciated by those of skill in the art that the present invention maybe realized without one or more of these details. In other examples,features and techniques known in the art will not be described forpurposes of brevity.

Many modern electronic devices have integrated imaging systems that canbe used for various features. For example, integrated imaging systemsare ubiquitous in smartphones, automated teller machines, physicalaccess control systems (e.g., electronic door locks), etc. In somecases, such imaging systems can provide user authentication features,such as for access control, biometric verification, and the like. Someimaging-based authentication features exploit face identification (faceID). For example, face identification can be used to provide depthand/or focusing information to the same and/or other imaging systems, toverify the authenticity or identification of a user, and/or for otherpurposes.

For the sake of context, FIG. 1 illustrates a mobile device 100 with anintegrated imaging system. The mobile device can be a smart phone, atablet computer, a laptop computer, or the like, according to someembodiments. The mobile device 100 may include a display screen 120, aframe 110 surrounding the display screen 120, control buttons (e.g., apower button 140, audio volume control buttons 130, a holding forcesensor 150, etc.), and/or any other components. The integrated imagingsystem can include one or more cameras 160, such as a front camera(e.g., a “selfie” camera), a rear camera, etc. In some implementations,the front camera 160 is also used for face ID sensing. In otherimplementations, a dedicated camera 160 is provided for face ID sensing.

FIG. 2 illustrates a simplified block diagram of an electronic accesscontrol system 200 using face ID sensing according to some embodiments.An imaging camera 210, such as the camera 160 of FIG. 1, can have anassociated field of view (FOV) within which it can image athree-dimensional (3D) object 202. In the context described herein, itis generally assumed that the 3D object 202 is a human face, or ispurported to be a human face. However, it will be appreciated that suchdescriptions should not be construed as limiting the invention tooperate only in context of human faces and/or face ID sensing; rather,embodiments described herein can be applied to any suitable imagingcontext.

The camera 210 captures an image 220 of the 3D object 202 using imagingoptics (e.g., lenses, mirrors, filters, etc.), sensors (e.g.,photodetectors), and/or any suitable components. In some embodiments,capturing of the image 220 can involve focusing the imaging opticsand/or tuning the sensors to form a clear image with desired sharpness,contrast, chromatic characteristics, etc. For example, the capturedimage 220 can have no or very small distortions or other types of imageaberrations (e.g., spherical aberration, coma, astigmatism, and fieldcurvature).

A control and processing module 230 may analyze the characteristics ofthe image 220. For example, the control and processing module 230 may beconfigured to identify that the image 220 contains an image of a humanface, and can extract facial signatures of the human face from the image220. The control and processing module 230 may be further configured tocompare the facial signatures in the image 220 with facial signatures ofan authorized user stored in a face ID database 240. The face IDdatabase may include face ID data of the authorized user generatedduring a registration process. For example, during the registrationprocess, one or more images of the live face of the authorized user maybe captured by the camera 210. The images may be analyzed to extract(e.g., and characterize, etc.) the facial signatures of the authorizeduser. The facial images of the authorized user, as well as the facialsignatures, may be stored in the face ID database 240 for subsequentsecurity check.

The control and processing module 230 may determine whether the facialsignatures in the ideal image 220 matches with a face ID data stored inthe face ID database 240. The control and processing module 230 mayoutput a facial recognition decision via an output interface 250. Aprocessing unit of the mobile device 100 may grant or deny access,and/or provide other features, based on the facial recognition decision.For example, if the control and processing module 230 outputs a positivefacial recognition decision indicating a match, the processing unit ofthe mobile device 100 of FIG. 1 may grant access to the mobile device100 (e.g., waking up the mobile device 100), authorize a paymenttransaction using the mobile device, etc. On the other hand, if thecontrol and processing module 230 outputs a negative facial recognitiondecision indicating a non-match, the processing unit of the mobiledevice 100 may deny access to the mobile device 100 (e.g., the mobiledevice 100 may remain locked).

In many practical contexts, such face ID sensing is designed to balancepotentially competing considerations. For example, conventionalapproaches tend to analyze the captured image 220 only to a level atwhich to a relatively small number of characteristic points ormeasurements can be extracted from large-scale facial structures (e.g.,corners of eyes). Those extracted points or measurements are thencompared against previously registered points or measurements todetermine whether a statistical match is apparent. Such an approach canbe relatively fast and light-weight (e.g., using minimal computational,memory, battery, and/or other resources), but may also provide arelatively low level of security. For example, such an approach mayyield a false-positive match for similar looking individuals and/or maybe relatively simple to spoof using a high-resolution two-dimensionalimage of an authorized user's face, a three-dimensional wax or latexmodel of an authorized user's face, or the like. For many smartphone andother applications, it can be desirable to provide a higher level ofsecurity (including additional protection against spoofing), withoutexcessively impacting battery, memory, processor, and other resources.

In general, embodiments described herein exploit various opticalprinciples, including image size, to derive depth information fromimaging. FIG. 3 shows an imaging environment 300 to illustrate theoptical principles involves in using image size to derive depthinformation from imaging, as a context for embodiments herein. Asillustrated, a lens 310 forms an image 325 of an object 320 on aphotodetector 305 (e.g., a photodiode array). The lens 310 has a knownfocal length (f) 315, the photodetector 305 is a known imaging distance(S′) 335 from the lens 310, and the object 320 has a known objectdimension (h) 340. When the object 320 is a particular object distance(S) 330 from the photodetector 305, the image 325 will manifest adimension (h′) 345 corresponding to h 340; S 330 is a function of h′ 345in context of the known values off 315, S′ 335, and h 340. As such, S330 can be computed by measuring h′ 345, as long as h′ 345 can be mappedto a known h 340.

In context of face ID sensing, a number of deterministicmacro-structures (i.e., large scale structures) have been shown toprovide measurements that are relatively characteristic of a particularindividual and tend to remain consistent over time for that particularindividual. FIG. 4 shows an illustrative image 400 of a face withvarious deterministic macro-structures. As illustrated, thedeterministic macro-structures often correspond to sensing organs of theface, such as eyes, nose, and ears. For example, the illustrated image400 indicates measurements of eye width 410 (e.g., between the outercorners of an eye), distance between eyes 420 (e.g., between the innercorners of the two eyes), iris diameter 415, distance from eye corner tobottom of nose 425, height of ears 430, width of nose 435, etc.Additional deterministic macro-structures can be used in someimplementations, such as sizes of front teeth, nose length, etc. Oncethese measurements are obtained, they can be compared to previouslyobtained measurements for purportedly the same face to determine whetherthere is a match. For example, as described above in context of FIG. 2,corresponding deterministic macro-structure measurements are obtainedduring a registration process and recorded in a face ID database 240,and presently obtained deterministic macro-structure measurements arecompared by the control and processing module 230 to determine whetherthere is a match. In some cases, the number of “profile” (i.e.,pre-registered) deterministic macro-structure measurements may bedifferent than the number of obtained deterministic macro-structuremeasurements. For example, during the registration process, a user maybe prompted to turn the head slightly in different ways, changeexpressions, etc. to allow collection of a relatively large set ofcandidate deterministic macro-structure measurements. However, duringsubsequent imaging (e.g., for user authentication, or the like), it isexpected that only a portion of those profile deterministicmacro-structure measurements are obtainable by the imaging system (e.g.,during imaging, the user's face is oriented so that certain features arehidden from the view of the imaging system, or the lighting is such thatcertain features are obscured, shadowed, etc.).

Some macro-structures tend not to be used in face ID sensing contextsbecause they are not sufficiently deterministic. For example, the mouthappreciably changes shape with changes in emotion and facial expression,such that the mouth tends not to provide deterministic macro-structuremeasurements. Similarly, eyebrows, forehead wrinkles, hairline, pupildiameter, and other large-scale structures of a face image are likely tochange from one imaging session to another. Further, some deterministicmacro-structures tend to be selected or excluded in face ID sensingcontexts based on how easily or reliably they can be measured. Forexample, conventional face ID sensing techniques may not be able toreliably locate the tip of a nose because there may not be an easilyidentifiable feature in that location, or conventional face ID sensingtechniques may not be able to reliably locate the tip of an earlobebecause the earlobe may not reliably be in view of the imaging system.

Different implementations and implementation contexts can yielddifferent approaches to obtaining deterministic macro-structuremeasurements. As a user turns or tilts her head relative to the imagingsystem, and/or changes her distance from the imaging system, certainmeasurements can change. Still, such effects on the measurements tend tobe substantially deterministic. For example, when a head is turned, eacheye is at a different distance from the imaging system and has adifferent 3D orientation with respect to the imaging system; but thoseeye-to-eye differences tend to follow predictable mathematical patterns.As such, rather than relying solely on the deterministic macro-structuremeasurements directly obtained from the image, embodiments can employadditional computations, normalizations, statistical processes, and/orother processes to account for these and other types of effects on thosemeasurements. For example, in some implementations, deterministicmacro-structure measurements include computationally derivedmeasurements. For example, an implementation can measure the distancebetween the eyes 420 and the distance from an eye corner to the bottomof the nose 425, and can further compute a ratio of measurements 420 and425 as a computationally derived measurement. In some embodiments, suchcomputationally derived measurements are used as part of the set ofdeterministic macro-structure measurements. In other embodiments, suchcomputationally derived measurements are used to correct the set ofdeterministic macro-structure measurements. For example, expecteddeterministic macro-structure measurements can be considered as lying inan expected measurement plane, and any changes to the orientation orlocation of the user's head effectively changes the orientation andlocation of the measurement plane to that of an imaged measurement plane(and correspondingly changes the positions and orientations of theobtained deterministic macro-structure measurements). Thecomputationally derived measurements can be used to mathematicallycharacterize the orientation and/or position of the imaged measurementplane, and to determine and apply a corresponding mathematical transformto reposition and reorient the obtained deterministic macro-structuremeasurements into the expected measurement plane.

As noted above, such deterministic macro-structure measurements onlytend to yield a relatively small number of data points. While such anapproach can be relatively fast and light-weight, it may be prone tooutputting false-positive matches for similar looking individuals, or tospoof attacks using high-resolution two-dimensional images of anauthorized user's face and/or using three-dimensional models of anauthorized user's face. Embodiments described herein exploit suchdeterministic macro-structure measurements to facilitate obtainingdeterministic micro-structure measurements. Such deterministicmicro-structure measurements can be exceedingly difficult (e.g.,practically impossible) to spoof.

FIG. 5 shows an illustrative set of imaging data 500 as a context forvarious embodiments described herein. The illustrative set of imagingdata 500 includes a high-definition image 510 of a portion of a humanface, and multiple characteristic sub-images 520, each associated with arespective deterministic micro-structure feature region. Theillustrative set of imaging data 500 is intended only to illustratefeatures of embodiments, and is not intended to limit the types ofimages described herein. For example, though the illustrative set ofimaging data 500 includes processed output images from imaging systems,some embodiments described herein rely on imaging data that includes rawoutput data from image sensors (e.g., that has not been color-corrected,or otherwise processed).

The “object” (i.e., the imaged portion of the human face) shown in image510 includes a number of different types of traceable structures. Asdescribed herein, embodiments can use these traceable structures toreliably locate deterministic micro-structure feature regions. In someembodiments, the traceable structures are the deterministicmacro-structures, such as described in FIG. 4. For example, thetraceable structures include eyes, ears, and nose. Such deterministicmacro-structures can yield deterministically measurable featurelocations, such as eye corners, eye width, nose width, etc. In someembodiments, the traceable structures can include additionalmacro-structures that are not necessarily deterministic, but may stillbe reliable locators for deterministic micro-structure feature regions.For example, the traceable structures in such embodiments can includeeyebrows, eyelashes, eyelids, nostrils, lips, etc.

The deterministic micro-structures can be small-scale structures of theimaged object that are sufficiently consistent from one imaging sessionto another to be useful for face identification. Such deterministicmicro-structures tend not to be easily seen or characterized withoutadditional image processing. In some embodiments, such deterministicmicro-structures are skin texture features, such as pore patterns.Notably, the deterministic micro-structures do not includenon-deterministic features. For example, freckle patterns may tend tochange over time with a person's recent sun exposure, or the like. Insome implementations, the deterministic micro-structures can includeother types of small-scale deterministic structures, such as iris veinpatterns, or the like. However, some such micro-structures, even thoughdeterministic, may still be prone to spoofing. For example, ahigh-resolution photograph may sufficiently capture vein patterns in aperson's eyes to spoof an imaging system (e.g., as opposed to skintextures, which may not be captured by even the highest-resolutionphotographs). As such, some embodiments avoid using those types ofdeterministic micro-structures for face ID sensing, or only use thosetypes of deterministic micro-structures along with other types ofdeterministic micro-structures that are less prone to spoofing.

The deterministic macro-structures can be used to locate deterministicmicro-structure feature regions in any suitable manner. For example, asillustrated, the deterministic macro-structure measurements can be usedto generate various grids, distances, angles, etc., from which to guidelocation of one or more deterministic micro-structure feature regions.As one example, a first deterministic micro-structure feature region isknown to be located (e.g., based on prior registration) some verticaldistance from the bottom of the nose. Upon imaging of the face, avertical reference is computed as running from a center between the eyesto a center of the chin; and the first deterministic micro-structurefeature regions can be found at the corresponding distance along thatvertical reference from the bottom of the nose. As such, a firstcharacteristic sub-image 520 a can be derived to correspond to the firstdeterministic micro-structure feature region. As another example, asecond deterministic micro-structure feature region is known to belocated at a point on the cheek that corresponding to a particularintersection of reference lines and triangles. In particular, uponimaging the face, a triangle is located with vertices at the bottomcenter-point of the nose, the outer-right eye corner, and the centerpoint of the chin; a horizontal reference line is located to passthrough the of the bottom center-point of the nose and to runperpendicular to the vertical reference; and the location of the seconddeterministic micro-structure feature region is derived from theintersection of the horizontal reference with the hypotenuse of thetriangle. As such, a second characteristic sub-image 520 b can bederived to correspond to the second deterministic micro-structurefeature region.

Having located the deterministic micro-structure feature regions, thecorresponding characteristic sub-images 520 at those locations can beprocessed to derive deterministic micro-structure measurements. FIG. 6shows illustrative partial chromatic responses 600 for illustrativeportions of two characteristic sub-images. In particular, a firstchromatic response 600 a is derived from a portion of characteristicsub-image 520 d of FIG. 5, which corresponds to a feature region aroundthe tip of the nose; and a second chromatic response 600 b is derivedfrom a portion of characteristic sub-image 520 b of FIG. 5, whichcorresponds to a feature region around the cheek. Each illustratedchromatic response 600 is an image brightness plot, shown as a plot ofbrightness value of the chromatic component (e.g., signal level ofcorresponding photodetector elements) versus location, over afifty-pixel-long row of an image sensor array. For example, the imagingsystem is focused using a medium-wavelength chromatic component, such asusing “green” chromatic responses, and the plotted value indicates themagnitude of response of a red photodiode component of each pixel of aphotodiode array of the imaging system. Some embodiments described canoperate regardless of which color component(s) are used for focusing,color correction, imaging, chromatic response plots, etc. For example,certain chromatic configuration options can tend to increase responsemagnitudes, improve contrast, and/or otherwise improve certain sensingparameters; but some embodiments may still operate reliably across alarge range of such chromatic configuration options.

Multiple types of information can be obtained from the chromaticresponses 600. To obtain such information, implementations can computestatistics to measure the distribution of brightness slopes, standarddeviations of brightness valley depths, profile valley widths, and/orother values. For example, an illustrative brightness valley depths 620and an illustrative profile valley width 610 are shown in each plot 600.Valley depth information can be denoted by the image contrast, andaverage valley width at an average valley depth can be computed to forma face structure map. Such plots and values can be generated andanalyzed (i.e., computations performed) across some or all portions ofthe image. For example, some implementations compute such values acrossan entire face image, and other implementations only compute such valueswithin predefined and located deterministic micro-structure featureregions. As described above, the various plots and values can be mappedto face locations in accordance with the deterministic macro-structuremeasurements and locations. In some embodiments, the deriveddeterministic micro-structure measurements are mapped to deterministicmacro-structure locations to establish a 3D map of an entire face orportions of a face.

It can be seen that the chromatic response plots 600 can be used toobtain (e.g., extract, derive, compute, etc.) textural signatures. Thepattern of peaks and valleys in the chromatic responses 600 can beindicative of deterministic micro-structures, for example, correspondingto the pores and/or other textural variations of the skin in therespective portions of the respective characteristic sub-images 520.Obtaining and characterizing such a textural signature can supportcertain features. One such feature is that sensing the presence of sucha textural signature clearly indicates that the images object is a 3Dobject with a pattern of micro-structures. For example, ahigh-resolution 2D photograph may match deterministic macro-structuresof a pre-registered individual. However, such a photograph will notinclude such micro-structures, and imaging of the photograph will notproduce such textural signatures. Notably, such a feature does not relyon pre-registration or matching of any particular textural signature;only that a textural signature is present. For example, animplementation can use deterministic macro-structures for face IDsensing, and can further detect presence of any textural signature toensure that the imaged object is not a 2D photograph, or a 3D modelwithout texture.

Another use of such textural signatures is to determine whether anobtained textural signature is characteristic of the object beingimaged. For example, patterns and/or ranges of measurements of valleydepths 620 and valley widths 610 obtained from human face skin imagesmay tend to be relatively consistent across most or all human faces. Assuch, some embodiments can determine whether a derived texturalsignature is characteristic of a human face, even withoutpre-registration or matching of any particular textural signature. Somesuch embodiments can, for example, use deterministic macro-structuresfor face ID sensing, and can further detect presence of a characteristictextural signature to indicate that the imaged object is a real humanface (i.e., not a 2D photograph, 3D model, or other spoof).

Another use of such textural signatures is to determine whether anobtained textural signature is characteristic of a particular,pre-registered user. For example, particular patterns or measurements ofvalley depths 620 and valley widths 610 obtained from characteristicsub-images 520 of a user's face are correspond to deterministicmicro-structures that are practically unique to the user (e.g.,sufficiently unique for use in facial recognition, user authentication,biometric verification, etc.). As such, some embodiments can determinewhether a derived textural signature matches a profile (i.e.,pre-registered) set of textural signatures for a user who is purportedlybeing imaged. In such embodiments, face ID sensing can use bothdeterministic macro-structures and deterministic micro-structures tosupport both verification of a user's identify and spoof detection.

While the illustrated plots only show a chromatic response plots for asingle chromatic component (i.e., red), chromatic information can yieldadditional information that can support additional features. In someembodiments, a single narrow-band optical wavelength is used for face IDsensing. For example, a particular wavelength is chosen to yield sharpcontrast in the chromatic response across a wide range of skin tones,pigments, and other characteristics. Some embodiments can use lightwithin the visible spectrum. Other embodiments can additionally oralternatively use light outside of the visible spectrum, such as in aninfrared (e.g., near-infrared), or other spectrum. In some embodiments,relative and/or absolute depth information can be obtained by comparingchromatic response data across multiple chromatic responses. Forexample, the imaging system can be focused according to a greenchromatic component, and chromatic response plots can be generated forred and blue chromatic components derived from imaging of an object atsome distance from the imaging system. Because the lens will tend tohave different focal lengths for different chromatic components atdifferent object distances, differences in sharpness indicated by thedifferent chromatic component responses for a particular characteristicsub-image 520 can be indicative of the relative (e.g., or absolute, ifcalibrated) distance of the deterministic micro-structure feature regioncorresponding to that characteristic sub-image 520. This depthinformation can be used for various purposes, such as to help determinethe orientation and/or position of the face being images, to help findone or more absolute reference distances, etc.

Some embodiments can be implemented with only relative distancemeasurements. As described above, some such embodiments can rely oncomputationally derived measurements, or the like. For example, uponimaging the face, 2D or 3D coordinate locations are recorded for thebottom center-point of the nose (A), the outer-right eye corner (B), andthe center point of the chin (C). In such a case, all these coordinatelocations may be referenced to some generated image reference coordinatesystem and may not have any relation to absolute measurements. Still, areference vertical can be generated according to line AC, a referencehorizontal can be generated to intersect perpendicularly with thereference vertical at point A, a reference triangle can be generated astriangle ABC, etc. Without any absolute distance measurements, thelocation of a deterministic micro-structure feature region can beobtained according to the reference features. As one example, thepre-registered location of a deterministic micro-structure featureregion can be defined as the end of a vector that originates at point Ain a direction that bisects line BC at some location (D) and extends1.4-times the distance of AD.

In other embodiments, it can be desirable to obtain one or more absolutemeasurements. In some such embodiments, calibrated chromaticdifferentiation can be used to derive at least one absolute depthmeasurement. In other such embodiments, measurements can be obtained(e.g., at least during registration) in context of a referencemeasurement guide. For example, the reference measurement guide can be aruler, a grid, a frame, a barcode, or anything for which the absolutesize is known to, or can be obtained by, the imaging system. In someimplementations, the reference measurement guide is implemented on atransparent substrate, such as a ruler printed on a transparent sticker.The reference measurement guide can be placed in one or more locations.For example, a subset of the deterministic macro-structures can bedefined for registration and calibration, and the reference measurementguide (or multiple reference measurement guides) can be placed in ornear those locations during the registration process.

FIG. 7 shows a flow diagram of an illustrative method 700 forregistration of a user, according to various embodiments herein. Atstage 704, a face ID registration routine is activated. For example,face ID registration may be activated by an authorized user of a mobiledevice by selecting face ID registration in “settings,” or when themobile device is turned on by the authorized user for the first timeafter purchase. At stage 708, the camera may capture a set of images ofthe authorized user's face. For example, the set of images can includeone or more images under one or more imaging conditions, such as usingone or more lighting conditions, one or more focus conditions, one ormore cameras, one or more aperture or other optical settings, one ormore orientations, etc. Similarly, the registration process can promptthe user to capture images of the user at different angles, fromdifferent distances, with the head turned in different ways, etc. Insome embodiments, capturing the images at stage 708 involves promptingthe user to keep eyes focused on one location, or otherwise to positionthe user's head, eyes, or other features according to specificinstructions. In some embodiments, one or more images captured in stage708 in context of a reference measurement guide, such as a ruler. Forexample, the user is prompted to place one or more reference measurementguide in the field of view of the imaging system, is a particularlocation with respect to one or more facial features (e.g., one or moredeterministic macro-structures), etc.

At stage 712, embodiments can process the images to extractdeterministic macro-structure (DMAS) measurements. For example, imageprocessing techniques can be used to identify large-scale facialfeatures, such as eyes, nose, mouth, etc.; and various deterministicmeasurements can be obtained from those features, such as eye cornerlocations, nose width, nose height, eye-to-eye spacing, etc. At stage716, some or all of the DMAS measurements can be stored in a face IDdatabase in association with the authorized user. For example, the faceID database can store one or more registration profiles for one or moreauthorized users, and the DMAS measurements can be stored in theregistration profile of the authorized user who activated theregistration process at stage 704.

At stage 720, embodiments can further process the images to identify andlocate deterministic micro-structure (DMIS) feature regions. In someembodiments, image processing techniques are used to identify regions ofthe face images most likely to include DMIS feature regions, such asrelatively large skin areas generally lacking in DMASs or othertraceable structures (e.g., skin regions of the cheeks, forehead, nose,etc.). Such embodiments can then determine a location definition foreach identified region based on DMAS measurement locations. As describedabove, in some implementations, the location definition is an algorithmthat locates the corresponding DMIS feature region based on a set ofreferences (e.g., reference lines, reference polygons, etc.) generatedfrom the DMAS measurements. Other implementations can use any othersuitable type of location definition. For example, the locationdefinition can indicate a set of coordinates in a reference coordinateplane that can be mathematically transformed (e.g., positioned,oriented, scaled, skewed, etc.) based on the DMAS measurements. In otherembodiments, the registration process 700 is pre-programmed withcandidate location definitions at which identifiable DMIS featureregions are likely to be found. Such embodiments can then seek toidentify DMIS feature regions using those location definitions. Asdescribed above, in some implementations, the location definition is analgorithm that locates the corresponding DMIS feature region based on aset of references (e.g., reference lines, reference polygons, etc.)generated from the DMAS measurements. Other implementations can use anyother suitable type of location definition. For example, the locationdefinition can indicate a set of coordinates in a reference coordinateplane that can be mathematically transformed (e.g., positioned,oriented, scaled, skewed, etc.) based on the DMAS measurements.

At stage 724, embodiments can obtain characteristic sub-images based onlocations in the face images identified in stage 720 as corresponding toDMIS feature regions. At stage 728, embodiments can compute DMISsignatures from the characteristic sub-images for at least a portion ofthe identified DMIS feature regions. In some embodiments, obtaining thecharacteristic sub-images at stage 724 comprises extracting a portion ofthe images captured in stage 708. For example, even though the faceimages captured in stage 708 may include imaging data for the entireface, sub-image portions of those images may still have enoughresolution to support computations of DMIS signatures in stage 728. Inother embodiments, one or more additional images is captured to form thecharacteristics sub-images. For example, the additional images can becaptured with different imaging conditions (e.g., different focussettings, aperture settings, lighting settings, zoom settings, etc.) tooptimize sharpness of the imaging data in the characteristic sub-imagearea. As described above, the computations at stage 728 can includestatistical processing, and/or other image processing, to identify DMIStextural signatures in the image data. For example, average peakheights, valley widths, and/or other data can be extracted fromchromatic response data to indicate micro-textures of the region, suchas due to pore patterns, and the like.

At stage 732, embodiments can store various data obtained in theregistration process as further registration profile data for theauthorized user in the face ID database. In some embodiments, for eachDMIS feature region (of some or all of the DMIS feature regions), thestoring at stage 732 includes storing the location definition for theDMIS feature region, and at least one DMIS signature computed for theDMIS feature region. For example, subsequent to completing theregistration process 700, a registration profile is stored for anauthorized user, and the registration profile includes at least: (a) aset of DMAS measurements; (b) a set of location definitions for DMISfeature regions, at least some defined according to the DMASmeasurements; and (c) a set of DMIS signatures computed for the DMISfeature regions.

FIG. 8 shows a flow diagram of an illustrative method 800 forauthentication of a user, according to various embodiments herein. Atstage 804, a face ID authentication routine is activated. For example,face ID authentication may be activated by a purportedly authorized userof a mobile device in connection to the user attempting to unlock thephone (from a locked state), to authorize a mobile payment transaction,to authorize user credentials for an application, etc. As describedherein, the face ID authentication can be configured for differentlevels of security, different levels of spoof detection, etc. Forexample, reducing the number of data points used for face IDauthentication, reducing the face ID authentication's reliance on DMISdata, and/or other factors can effectively reduce the security providedby the face ID authentication. To that end, some embodiments can betuned for increased convenience in exchange for reduced security, forexample, by reducing the chance of false negatives (e.g., to reducefrustration of a user repeatedly being unable to access her device), butcorrespondingly increasing the number of false positives (e.g.,potentially authenticating users who are not authorized). Similarly,some embodiments can be tuned for added security, regardless of impactson convenience. For example, embodiments can be configured toappreciably reduce the chance of false positives by increasing theamount of DMAS and DMIS data used for authentication, even though suchan increase can potentially yield more frustrating false negatives, andmay increase resource burdens associated with the face ID authentication(e.g., burdens to memory, battery, processor, and/or other resources).

At stage 808, the camera may capture a set of images of the authorizeduser's face. In some embodiments, the set of images can be collected insubstantially the same manner as in stage 708 of the registrationprocess 700. In other embodiments, the image capture at stage 808involves only a rapid capture of a single, or small number of, imageframes. For example, the imaging in stage 708 is more proactive (e.g.,collecting multiple images under multiple conditions, with prompts tothe user, etc.), while the imaging at stage 808 passively collects whatis in the field-of-view of the imaging system. For example, poor imagecapturing at stage 808 can simply result in a denial of authenticationfor the user; and embodiments may iterate the method 800 untilsufficient image data is captured to make the proper computations,computations, etc. for face ID authentication in accordance with themethod 800.

At stage 812, embodiments can process the images to extractdeterministic macro-structure (DMAS) measurements. In some embodiments,stage 812 is performed in substantially the same way as stage 712. Inother embodiments, stage 812 is performed in a manner that uses profileDMAS measurements as a guide. In some such embodiments, the image(s)from stage 808 may not be captured from optimal angles, or otherwiseunder optimal conditions. As such, extraction of the DMAS measurementsin stage 812 may involve image pre-processing, such as adjusting thelighting, color, scale, skew, contrast, rotation, or other parameters ofthe image(s). In other such embodiments, the profile DMAS measurementsfrom stage 712 can be used to estimate locations at which DMASmeasurements are likely to be found in stage 812.

For example, prior to, concurrent with, or otherwise in connection withthe processing in stage 812, one or more registration profiles can beretrieved at stage 816 from a face ID registration database. In someembodiments, activation of the face ID authentication process at stage804 includes identification of a purported user. For example, a userattempting to be authenticated has already identified herself in someother way, such as by entering a user name, code, password, etc. In suchcases, even if multiple authenticated users have registration profilesstored in the database, the method 800 may only need to retrieve and usethe one associated with the user seeking authentication. In otherembodiments, there may be only a single registration profile stored inthe database.

At stage 820, a first authentication stage can be performed in which theDMAS measurements extracted in stage 812 are compared with the profileDMAS measurements from stage 712. If the comparison at stage 820indicates a match, the process 800 may proceed with furtherauthentication. If the comparison at stage 820 indicates no match, theprocess 800 may end by denying authentication at stage 822. Differentimplementations can consider a match in different ways. For example,some implementations can consider the DMAS measurements as being a matchwhen a statistical correlation between the pre-registered dataset andthe newly acquired dataset is higher than a predetermined minimumthreshold value across the entire dataset. In other implementations,each DMAS measurement, or groupings of DMAS measurements, are evaluatedto determine whether each has a high enough statistical correlation withits counterpart data from the registration profile.

In some embodiments, determining whether there is a match at stage 820can involve additional computations and/or processing, such as applyinga three-dimensional transform to one or both datasets, converting fromone measurement base to another (e.g., from relative values to absolutevalues, or vice versa), etc. In some embodiments, the generating thefirst-level authentication determination at stage 820 is based oncomparing an integer number (N) of authentication DMAS measurements(e.g., 20) to N corresponding profile DMAS measurements to determinewhether there is at least a threshold magnitude of statisticalcorrelation (e.g., 95%). In some such embodiments, N and/or thethreshold magnitude is tunable. For example, a settings portal can beprovided by which an authorized user, an administrator, a provider, etc.can adjust the number of DMAS measurement points to collect and/or use,and/or to adjust the threshold correlation value for determining amatch.

If a match is determined to exist at stage 820, embodiments of themethod 800 can proceed to stage 824. At stage 824, embodiments canobtain characteristic sub-images based on locations in the face imagesthat correspond to DMIS feature regions. For example, as described withreference to the registration method 700, the registration profile caninclude location definitions for each of multiple DMIS feature regions.Embodiments can apply the location definitions to the images obtained instage 808 to locate DMIS feature regions, and can obtain characteristicsub-images for those feature regions, accordingly. Some such embodimentslocate each DMIS feature region of the one or more DMIS feature regionsas a function of applying the authentication DMAS measurements to arespective one of the profile location definitions. Each of thecharacteristic sub-images can then be obtained by extracting image datacorresponding to a defined polygonal region of pixels of the set ofimages in accordance with the locating. For example, the definedpolygonal region of pixels can include a 50-pixel long rectangle, or anysuitable region. As in stage 724 of the registration process 700,obtaining the characteristic sub-images at stage 824 can includeextracting a portion of the images captured in stage 808, and/orcapturing one or more additional images (e.g., under different imagingconditions).

At stage 828, embodiments can compute DMIS signatures from thecharacteristic sub-images for at least a portion of the identified DMISfeature regions. In some embodiments, the computations of stage 828 aresubstantially identical to those of stage 728. As described herein, thecomputations at stage 828 can include statistical processing, and/orother image processing, to identify DMIS textural signatures in theimage data. For example, average peak heights, valley widths, and/orother data can be extracted from chromatic response data to indicatemicro-textures of the region, such as due to pore patterns, and thelike. In some embodiments, the computing at stage 828 includes:generating a chromatic response plot from at least one of thecharacteristic sub-images corresponding to a respective one of the DMISfeature regions; computing a set of peak/valley data (e.g., peakheights, valley widths, peak spacing, averages of peak and/or valleymeasurements, etc.) for the respective one of the DMIS feature regionsbased on statistically processing the chromatic response plot; andanalyzing the peak/valley data to obtain the one or more authenticationDMIS signatures as indicating a textural signature of the respective oneof the DMIS feature regions.

At stage 832, a second authentication stage can be performed in whichthe DMIS signatures computed in stage 828 are compared with the profileDMIS signatures from stage 728 (and stored as part of the registrationprofile). In some embodiments, generating the second-levelauthentication determination at stage 832 can include: determining notto deny authentication for the user when the authentication DMISsignatures match the profile DMIS signatures to at least a thresholdstatistical correlation level (e.g., 95%); and determining to denyauthentication for the user otherwise. If the comparison at stage 832indicates a match, the process 800 may conclude by authenticating theuser at stage 836. If the comparison at stage 832 indicates no match,the process 800 may end by denying authentication at stage 822. In someembodiments, the determination at stage 832 is performed in a similarmanner to the determination at stage 820. In other embodiments,different parameters and/or techniques can be used in stage 832 todetermine whether there is a match.

Some implementations can be tuned to find a match at stage 820 only whenthere is a very high statistical correlation with the registrationprofile data, but a lower threshold is set for the determination atstage 832. For example, such tuning can support a configuration in whichthe primary face ID authentication is based on the determination atstage 820, and the determination at stage 832 is supplemented foranti-spoofing, and/or as a second verification of the determination atstage 820. Other implementations can be tuned to find a match at stage820 even with a relatively low statistical correlation with theregistration profile data, but will only authenticate a user if there isalso a very high correlation with registration profile data at stage832. For example, such tuning can support a configuration in which thedetermination at stage 820 is used as a quick initial check (e.g., avery lightweight initial check that can be iterated continuously as partof a lock mode background process, or the like), and the primary face IDauthentication is based on the determination at stage 832. Other tuningsare possible to support other contexts and features. In someembodiments, generating the first-level authentication determination instage 820 is based on comparing an integer number (N) of authenticationDMAS measurements to N corresponding profile DMAS measurements todetermine whether there is at least a first threshold magnitude ofstatistical correlation; generating the second-level authenticationdetermination at stage 832 is based on comparing an integer number (M)of authentication DMIS signatures to M corresponding profile DMISsignatures to determine whether there is at least a second thresholdmagnitude of statistical correlation; and at least one of N, M, thefirst threshold magnitude, or the second threshold magnitude is tunableby an authenticated user. In such embodiments, N and M can be equal ordifferent, and the threshold magnitudes can be the same or different.

FIG. 9 shows a flow diagram of an example gated process 900 for accesscontrol, according to various embodiments. For example, a user can seekaccess to a smart phone having an integrated image sensor system, suchas described herein. Access to the smart phone is locked until the usersuccessfully passes a biometric verification of the user's face based onpreviously registered data. Biometric verification (or authentication)generally refers to verifying biometrics of a candidate user (or otherobject) against corresponding biometrics of a previously registereduser. Biometric verification can be much simpler than so-calledbiometric identification. For example, biometric identification may seekto determine the identity of a candidate user from a general populationof users, such as my determining whether a fingerprint matches any of alarge database of fingerprints to at least some threshold confidencelevel; while biometric verification can begin with an assumed set (e.g.,one or a relatively small number) of pre-registered users, and can seekto determine whether a current candidate user seems to match one of theassumed set of users to a threshold level of confidence. Biometricaccess control systems, like those of the example smart phone, aretypically based on biometric verification. For example, the smart phone(or similarly, an identification badge, an electronic door lock, etc.)may only be associated with a single authorized user, and the functionof the system is to determine whether a candidate user attempting accessappears (e.g., statistically) to be the authorized user. Such a functiondoes not require the system to search a huge database in an attempt toidentify the candidate user.

In a pre-biometric trigger phase 910, embodiments can wait to detect acandidate image or images, which can trigger further biometricverification. For example, the image sensor can continuously,periodically, or otherwise obtain images. The images can be dynamicallyprocessed to detect a set of image data that is generally characteristicof a face, or otherwise of a candidate for biometric verification. Forexample, certain traceable structures are detected in a particularpattern (e.g., at relative locations, sizes, etc.) that indicate to thesystem that the captured image is a candidate face image for biometricprocessing. In some implementations, this phase 910 can use varioustechniques to improve the detection of such traceable structures. Forexample, the phase 910 can include focusing the imaging system based onone or more parameters, such as based on a chromatic component; and/orthe phase 910 can include analysis of individual chromatic components ofthe raw image data (e.g., including computing statistical analyses ofimage brightness plots, etc.); and/or the phase 910 can involvecorrecting imaging data for parameters, such as contrast, spectrumreflectance, spectrum illumination inequality, surface transmission,etc.

In a biometric verification phase 920, the same and/or differenttraceable structures are used for biometric verification of thepre-registered user. In some implementations, the imaging data obtainedin phase 910 is sufficient for the biometric verification in phase 920.In other implementations, additional and/or different imaging data isobtained, such as high-definition data with multiple chromaticcomponents. In some embodiments, the phase 920 can involve resizingand/or reorienting the obtained data, and/or correcting the data forsize and/or orientation. For example, as described above, certaintraceable structures have known sizes, certain distances are knownbetween traceable structures, etc. Comparing such known information tothe obtained information can provide information as to the distance ofthe imaged object from the imaging system (e.g., objects appear smalleras they move farther from the imaging system), and/or to the orientationof the imaged object with respect to the imaging system (e.g., when theimaged object is tilted, its set of traceable structures is tilted in adeterministic manner). In some implementations, parameters of theimaging system are also known and can be used in this phase 920. Forexample, correlations between size and distance can be a function ofcertain lens parameters, focusing data, etc. As described above, thebiometric verification can be based on determining whether the traceablestructures (e.g., sizes, locations, separations, shapes, etc.) appear tomatch those of the registered object. Further as described above,biometric verification in phase 920 can additionally or alternatively bebased on textural signatures being characteristic of a registered user.For example, the biometric verification can determine matches betweenpresent imaging data and pre-registered imaging data based on some orall of: the set of DMAS measurements; the set of location definitionsfor DMIS feature regions, at least some defined according to the DMASmeasurements; and the set of DMIS signatures computed for the DMISfeature regions.

Some embodiments end with successful passage of the biometricverification phase 920. For example, passing the biometric verificationphase 920 triggers output of a verification signal, which triggers anaccess control system to permit access at phase 940 (e.g., the smartphone unlocks). Other embodiments further include a spoof detectionphase 930. For example, successful passage of the biometric verificationin phase 920 can trigger a final hurdle of the spoof detection phase930, which must also be passed prior to permitting access by an accesscontrol system at stage 940. As described above, such a spoof detectionphase 930 can use information obtained in the biometric verificationphase 920, and/or can obtain any suitable information, to determinewhether the candidate object is a spoof. For example, DMIS signaturescan be indicative of whether or not a purported human face of a user isa real face, or some type of spoof (e.g., a high-resolution 2Dphotograph, a 3D sculpture of a face in wax or latex, mask prostheticsbeing worn over a human face, etc.).

Some embodiments may include only one or two of the phases of the flowdiagram 900, and the various phases can be performed in any order. Insome embodiments, the spoof detection phase 930 and the biometricverification phase 920 are performed sequentially. For example,successful passage of the biometric verification phase 920 triggers astart of the spoof detection phase 930. In other embodiments, thebiometric verification phase 920 and the spoof detection phase 930 areperformed concurrently (i.e., at least partially in parallel). In someembodiments, some or all phases can be independently triggered. Forexample, a user can explicitly trigger a biometric verification phase920, such that the phase 920 is not responsive to successfulidentification of a candidate in phase 910. Similarly, a user canexplicitly trigger a spoof detection phase 930 without an associatedbiometric verification phase 920. For example, there may be an instancewhere the user desires to know whether an object is a spoof withoutdetermining any type of biometric verification of the object.

FIG. 10 provides a schematic illustration of one embodiment of acomputer system 1000 that can implement various system components and/orperform various steps of methods provided by various embodiments. Itshould be noted that FIG. 10 is meant only to provide a generalizedillustration of various components, any or all of which may be utilizedas appropriate. FIG. 10, therefore, broadly illustrates how individualsystem elements may be implemented in a relatively separated orrelatively more integrated manner.

The computer system 1000 is shown including hardware elements that canbe electrically coupled via a bus 1005 (or may otherwise be incommunication, as appropriate). The hardware elements may include one ormore processors 1010, including, without limitation, one or moregeneral-purpose processors and/or one or more special-purpose processors(such as digital signal processing chips, graphics accelerationprocessors, video decoders, and/or the like). For example, processors1010 can implement control and processing module 230 shown in FIG. 2.Some embodiments include one or more input/output (I/O) devices 1015. Insome implementations, the I/O devices 1015 include human-interfacedevices, such as buttons, switches, keypads, indicators, displays, etc.In other implementations, the I/O devices 1015 include circuit-leveldevices, such as pins, dip-switches, etc. In some implementations, thecomputer system 1000 is a server computer configured to interface withadditional computers and/or devices, such that the I/O devices 1015include various physical and/or logical interfaces (e.g., ports, etc.)to facilitate hardware-to-hardware coupling, interaction, control, etc.In some embodiments, the I/O devices 1015 implement output interface 250of FIG. 2.

The computer system 1000 may further include (and/or be in communicationwith) one or more non-transitory storage devices 1025, which cancomprise, without limitation, local and/or network accessible storage,and/or can include, without limitation, a disk drive, a drive array, anoptical storage device, a solid-state storage device, such as a randomaccess memory (“RAM”), and/or a read-only memory (“ROM”), which can beprogrammable, flash-updateable and/or the like. Such storage devices maybe configured to implement any appropriate data stores, including,without limitation, various file systems, database structures, and/orthe like. In some embodiments, the storage devices 1025 include face IDregistration database 240 of FIG. 2.

The computer system 1000 can also include, or be in communication with,any other components described herein. In some embodiments, the computersystem 1000 includes an imaging subsystem 1020. The imaging subsystem1020 can include the imaging components 210 of FIG. 2, and/or any othersupporting components for imaging, as described herein. In someembodiments, the computer system 1000 includes an illumination subsystem1030. The illumination subsystem 1030 can include any suitableillumination sources for projecting normal illumination and/or referenceillumination into a field of view of the imaging subsystem 1020, and anysupporting components. In some such embodiments, the illuminationsubsystem 1030 includes one or more of illumination sources to providereference illumination flooding and/or to provide one or more types ofprobe illumination. Some embodiments can include additional subsystems,such as a communications subsystem (not shown) to communicatively couplewith other systems, networks, etc.

Embodiments of the computer system 1000 can further include a workingmemory 1035, which can include a RAM or ROM device, as described herein.The computer system 1000 also can include software elements, shown ascurrently being located within the working memory 1035, including anoperating system 1040, device drivers, executable libraries, and/orother code, such as one or more application programs 1045, which mayinclude computer programs provided by various embodiments, and/or may bedesigned to implement methods, and/or configure systems, provided byother embodiments, as described herein. Merely by way of example, one ormore procedures described with respect to the method(s) discussed hereincan be implemented as code and/or instructions executable by a computer(and/or a processor within a computer); in an aspect, then, such codeand/or instructions can be used to configure and/or adapt a generalpurpose computer (or other device) to perform one or more operations inaccordance with the described methods. A set of these instructionsand/or codes can be stored on a non-transitory computer-readable storagemedium, such as the non-transitory storage device(s) 1025 describedabove. In some cases, the storage medium can be incorporated within acomputer system, such as computer system 1000. In other embodiments, thestorage medium can be separate from a computer system (e.g., a removablemedium, such as a compact disc), and/or provided in an installationpackage, such that the storage medium can be used to program, configure,and/or adapt a general purpose computer with the instructions/codestored thereon. These instructions can take the form of executable code,which is executable by the computer system 1000 and/or can take the formof source and/or installable code, which, upon compilation and/orinstallation on the computer system 1000 (e.g., using any of a varietyof generally available compilers, installation programs,compression/decompression utilities, etc.), then takes the form ofexecutable code.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware can also be used, and/or particularelements can be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices, such as network input/output devices, may beemployed.

As mentioned above, in one aspect, some embodiments may employ acomputer system (such as the computer system 1000) to perform methods inaccordance with various embodiments of the invention. According to a setof embodiments, some or all of the procedures of such methods areperformed by the computer system 1000 in response to processor 1010executing one or more sequences of one or more instructions (which canbe incorporated into the operating system 1040 and/or other code, suchas an application program 1045) contained in the working memory 1035.Such instructions may be read into the working memory 1035 from anothercomputer-readable medium, such as one or more of the non-transitorystorage device(s) 1025. Merely by way of example, execution of thesequences of instructions contained in the working memory 1035 can causethe processor(s) 1010 to perform one or more procedures of the methodsdescribed herein.

The terms “machine-readable medium,” “computer-readable storage medium”and “computer-readable medium,” as used herein, refer to any medium thatparticipates in providing data that causes a machine to operate in aspecific fashion. These mediums may be non-transitory. In an embodimentimplemented using the computer system 1000, various computer-readablemedia can be involved in providing instructions/code to processor(s)1010 for execution and/or can be used to store and/or carry suchinstructions/code. In many implementations, a computer-readable mediumis a physical and/or tangible storage medium. Such a medium may take theform of a non-volatile media or volatile media. Non-volatile mediainclude, for example, optical and/or magnetic disks, such as thenon-transitory storage device(s) 1025. Volatile media include, withoutlimitation, dynamic memory, such as the working memory 1035.

Common forms of physical and/or tangible computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, any other opticalmedium, any other physical medium with patterns of marks, a RAM, a PROM,EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any othermedium from which a computer can read instructions and/or code. Variousforms of computer-readable media may be involved in carrying one or moresequences of one or more instructions to the processor(s) 1010 forexecution. Merely by way of example, the instructions may initially becarried on a magnetic disk and/or optical disc of a remote computer. Aremote computer can load the instructions into its dynamic memory andsend the instructions as signals over a transmission medium to bereceived and/or executed by the computer system 1000.

It will be understood that, when an element or component is referred toherein as “connected to” or “coupled to” another element or component,it can be connected or coupled to the other element or component, orintervening elements or components may also be present. In contrast,when an element or component is referred to as being “directly connectedto,” or “directly coupled to” another element or component, there are nointervening elements or components present between them. It will beunderstood that, although the terms “first,” “second,” “third,” etc. maybe used herein to describe various elements, components, these elements,components, regions, should not be limited by these terms. These termsare only used to distinguish one element, component, from anotherelement, component. Thus, a first element, component, discussed belowcould be termed a second element, component, without departing from theteachings of the present invention. As used herein, the terms “logiclow,” “low state,” “low level,” “logic low level,” “low,” or “0” areused interchangeably. The terms “logic high,” “high state,” “highlevel,” “logic high level,” “high,” or “1” are used interchangeably.

As used herein, the terms “a”, “an” and “the” may include singular andplural references. It will be further understood that the terms“comprising”, “including”, having” and variants thereof, when used inthis specification, specify the presence of stated features, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, steps, operations,elements, components, and/or groups thereof. In contrast, the term“consisting of” when used in this specification, specifies the statedfeatures, steps, operations, elements, and/or components, and precludesadditional features, steps, operations, elements and/or components.Furthermore, as used herein, the words “and/or” may refer to andencompass any possible combinations of one or more of the associatedlisted items.

While the present invention is described herein with reference toillustrative embodiments, this description is not intended to beconstrued in a limiting sense. Rather, the purpose of the illustrativeembodiments is to make the spirit of the present invention be betterunderstood by those skilled in the art. In order not to obscure thescope of the invention, many details of well-known processes andmanufacturing techniques are omitted. Various modifications of theillustrative embodiments, as well as other embodiments, will be apparentto those of skill in the art upon reference to the description. It istherefore intended that the appended claims encompass any suchmodifications.

Furthermore, some of the features of the preferred embodiments of thepresent invention could be used to advantage without the correspondinguse of other features. As such, the foregoing description should beconsidered as merely illustrative of the principles of the invention,and not in limitation thereof. Those of skill in the art will appreciatevariations of the above-described embodiments that fall within the scopeof the invention. As a result, the invention is not limited to thespecific embodiments and illustrations discussed above, but by thefollowing claims and their equivalents.

What is claimed is:
 1. A method for authentication of a user based onpassive face imaging, the method comprising: capturing, by an imagingsystem, a set of images of a user's face; processing the set of imagesto extract authentication deterministic macro-structure (DMAS)measurements; generating a first-level authentication determinationbased on comparing the authentication DMAS measurements with profileDMAS measurements; obtaining one or more characteristic sub-images ofone or more deterministic micro-structure (DMIS) feature regions basedon profile location definitions for each of the one or more DMIS featureregions; computing one or more authentication DMIS signatures from theone or more characteristic sub-images; generating a second-levelauthentication determination based on comparing the authentication DMISsignatures with profile DMIS signatures computed for each of the one ormore DMIS feature regions; and outputting an authentication result basedon both the first-level authentication determination and thesecond-level authentication that indicates whether authentication of theuser is granted or denied.
 2. The method of claim 1, further comprising:retrieving a registration profile from a face identification database,the registration profile comprising the profile DMAS measurements, theprofile location definitions for each of the one or more DMIS featureregions, and the at least one profile DMIS signature computed for eachof the one or more DMIS feature regions.
 3. The method of claim 1,wherein: the first-level authentication determination indicates whetheror not to deny authentication of the user; and the obtaining, computing,and generating are performed only responsive to the first-levelauthentication determination indicating not to deny authentication ofthe user.
 4. The method of claim 1, wherein the generating thefirst-level authentication determination comprises: determining aregistration measurement base based on the profile DMAS measurements;determining an authentication measurement base based on theauthentication DMAS measurements; applying a mathematical transform sothat the profile DMAS measurements and the authentication DMASmeasurements are in a shared measurement base; and comparing theauthentication DMAS measurements with profile DMAS measurements in theshared measurement base.
 5. The method of claim 1, wherein thegenerating the first-level authentication determination comprises:determining not to deny authentication for the user when theauthentication DMAS measurements match the profile DMAS measurements toat least a threshold statistical correlation level; and determining todeny authentication for the user otherwise.
 6. The method of claim 1,wherein: the generating the first-level authentication determination isbased on comparing an integer number (N) of authentication DMASmeasurements to N corresponding profile DMAS measurements to determinewhether there is at least a threshold magnitude of statisticalcorrelation; and N and/or the threshold magnitude is tunable by anauthenticated user.
 7. The method of claim 1, wherein the obtaining theone or more characteristic sub-images comprises: locating each DMISfeature region of the one or more DMIS feature regions as a function ofapplying the authentication DMAS measurements to a respective one of theprofile location definitions; and obtaining each of the one or morecharacteristic sub-images by extracting image data corresponding to adefined polygonal region of pixels of the set of images in accordancewith the locating.
 8. The method of claim 1, wherein the computing theone or more authentication DMIS signatures from the one or morecharacteristic sub-images comprises: generating a chromatic responseplot from at least one of the characteristic sub-images corresponding toa respective one of the DMIS feature regions; computing a set ofpeak/valley data for the respective one of the DMIS feature regionsbased on statistically processing the chromatic response plot; andanalyzing the peak/valley data to obtain the one or more authenticationDMIS signatures as indicating a textural signature of the respective oneof the DMIS feature regions.
 9. The method of claim 1, wherein thegenerating the second-level authentication determination comprises:determining not to deny authentication for the user when theauthentication DMIS signatures match the profile DMIS signatures to atleast a threshold statistical correlation level; and determining to denyauthentication for the user otherwise.
 10. The method of claim 1,wherein: the generating the first-level authentication determination isbased on comparing an integer number (N) of authentication DMASmeasurements to N corresponding profile DMAS measurements to determinewhether there is at least a first threshold magnitude of statisticalcorrelation; the generating the second-level authenticationdetermination is based on comparing an integer number (M) ofauthentication DMIS signatures to M corresponding profile DMISsignatures to determine whether there is at least a second thresholdmagnitude of statistical correlation; and at least one of N, M, thefirst threshold magnitude, or the second threshold magnitude is tunableby an authenticated user.
 11. A system for authentication of a userbased on passive face imaging, the system comprising: an imaging camerato capture a set of images of a user's face; a face identificationdatabase having, stored thereon, a registration profile comprisingprofile deterministic macro-structure (DMAS) measurements, profilelocation definitions for each of one or more deterministicmicro-structure (DMIS) feature regions, and at least one profile DMISsignature computed for each of the one or more DMIS feature regions; anda control and processing module having one or more processors, and amemory having, stored thereon, a set of instructions which, whenexecuted, cause the one or more processors, operating in a userauthentication mode, to: process the set of images to extractauthentication DMAS measurements; generate a first-level authenticationdetermination based on comparing the authentication DMAS measurementswith the profile DMAS measurements; obtain one or more characteristicsub-images of the one or more DMIS feature regions based on the profilelocation definitions for each of the one or more DMIS feature regions;compute one or more authentication DMIS signatures from the one or morecharacteristic sub-images; generate a second-level authenticationdetermination based on comparing the authentication DMIS signatures withthe profile DMIS signatures; and output an authentication result basedon both the first-level authentication determination and thesecond-level authentication that indicates whether authentication of theuser is granted or denied.
 12. The system of claim 11, wherein the setof instructions, when executed, cause the one or more processorsfurther, operating in a registration mode, to: direct capturing, by theimaging camera, of a set of profile images of an authorized user's face;process the set of profile images to extract the profile DMASmeasurements and to identify the one or more profile DMIS featureregions; for each profile DMIS feature region of the one or more profileDMIS feature regions: generate a respective profile location definitionbased on the profile DMAS measurements; obtain one or more respectivecharacteristic profile sub-images from image data of the profile DMISfeature region; and computing one or more respective profile DMISsignatures from the one or more respective characteristic profilesub-images; and storing, as a registration profile for the authorizeduser in the face identification database, at least some of the profileDMAS measurements, at least some of the respective profile locationdefinitions for the profile DMIS feature regions, and at least some ofthe profile DMIS signatures computed for the profile DMIS featureregions.
 13. The system of claim 11, wherein: the first-levelauthentication determination indicates whether or not to denyauthentication of the user; and the set of instructions, when executed,cause the one or more processors to obtain the one or morecharacteristic sub-images, to compute the one or more authenticationDMIS signatures, and to generate the second-level authentication onlyresponsive to the first-level authentication determination indicatingnot to deny authentication of the user.
 14. The system of claim 11,wherein the set of instructions, when executed, cause the one or moreprocessors to generate the first-level authentication determination by:determining a registration measurement base based on the profile DMASmeasurements; determining an authentication measurement base based onthe authentication DMAS measurements; applying a mathematical transformso that the profile DMAS measurements and the authentication DMASmeasurements are in a shared measurement base; and comparing theauthentication DMAS measurements with profile DMAS measurements in theshared measurement base.
 15. The system of claim 11, wherein the set ofinstructions, when executed, cause the one or more processors togenerate the first-level authentication determination by: determiningnot to deny authentication for the user when the authentication DMASmeasurements match the profile DMAS measurements to at least a thresholdstatistical correlation level; and determining to deny authenticationfor the user otherwise.
 16. The system of claim 11, wherein the set ofinstructions, when executed, cause the one or more processors to obtainthe one or more characteristic sub-images by: locating each DMIS featureregion of the one or more DMIS feature regions as a function of applyingthe authentication DMAS measurements to a respective one of the profilelocation definitions; and obtaining each of the one or morecharacteristic sub-images by extracting image data corresponding to adefined polygonal region of pixels of the set of images in accordancewith the locating.
 17. The system of claim 11, wherein the set ofinstructions, when executed, cause the one or more processors to computethe one or more authentication DMIS signatures from the one or morecharacteristic sub-images by: generating a chromatic response plot fromat least one of the characteristic sub-images corresponding to arespective one of the DMIS feature regions; computing a set ofpeak/valley data for the respective one of the DMIS feature regionsbased on statistically processing the chromatic response plot; andanalyzing the peak/valley data to obtain the one or more authenticationDMIS signatures as indicating a textural signature of the respective oneof the DMIS feature regions.
 18. The system of claim 11, wherein the setof instructions, when executed, cause the one or more processors togenerate the second-level authentication determination by: determiningnot to deny authentication for the user when the authentication DMISsignatures match the profile DMIS signatures to at least a thresholdstatistical correlation level; and determining to deny authenticationfor the user otherwise.
 19. The system of claim 11, wherein: the set ofinstructions, when executed, cause the one or more processors togenerate the first-level authentication determination is based oncomparing an integer number (N) of authentication DMAS measurements to Ncorresponding profile DMAS measurements to determine whether there is atleast a first threshold magnitude of statistical correlation; the set ofinstructions, when executed, cause the one or more processors togenerate the second-level authentication determination is based oncomparing an integer number (M) of authentication DMIS signatures to Mcorresponding profile DMIS signatures to determine whether there is atleast a second threshold magnitude of statistical correlation; and atleast one of N, M, the first threshold magnitude, or the secondthreshold magnitude is tunable by an authenticated user.
 20. The systemof claim 11, further comprising: an electronic access control systemhaving the imaging camera, the control and processing module, and theface identification database integrated therein, wherein the electronicaccess control system is configured to permit or deny access to aphysical or electronic resource based on the authentication result.